Method and system for certifying operators of x-ray inspection systems

ABSTRACT

A method and system for using Threat Imaging Projection (TIP) technology, and the operator testing data that it generates, in order to certify x-ray inspection system operators and thereby assure an acceptable and uniform level of operator performance. In a preferred configuration, the operator views x-ray images of objects on a screening system equipped with TIP technology. When the operator believes that a threat item image appears on the screening system, the operator indicates such via an indicating means. The system is capable of recording, storing and transmitting individual operator performance data relating to proper detections, missed detections, and false alarms, i.e., when an operator indicates that a threat item image appears when no such image actually appears on the screening system. The system preferably contains a library of various threat item images, e.g., guns, bombs, knives, etc., which are classified according to type of threat and difficulty of detection.

[0001] This application claims priority to provisional applicationSerial No. 60/308,510, filed Jul. 27, 2001, which is hereby incorporatedby reference.

FIELD OF THE INVENTION

[0002] The field of the invention generally relates to x-ray inspectionsystems used for security purposes. More particularly, the inventionrelates to a method and system for testing and certifying operators ofsuch systems.

BACKGROUND OF THE INVENTION

[0003] X-ray inspection systems, e.g., baggage scanners, are commonlyused to detect and prevent the passage of prohibited items beyond asecurity checkpoint. Such x-ray inspection systems are typically used atairports, courthouses and other locations where security is necessary ordesirable. These inspection systems guard against items such as weaponsand explosives from entering the restricted area. Generally, objects areplaced on a conveyor and pass through the system at which time they arex-rayed. An x-ray image of the object appears on a monitor and is viewedby an operator.

[0004] Regardless of the technology used in x-ray inspection systems,the performance of such systems in large part relies on the vigilanceand skillfulness of the system operators who examine x-ray images of thecontents of baggage on the system's video monitor. In turn, thevigilance and skillfulness of the system operators generally depend onthe amount and type of training they receive. This is very importantsince operator performance is critical to the overall integrity ofsecurity inspection systems.

[0005] Traditional methods of training and certifying system operatorshave involved training and testing in a classroom environment, with theuse of photographs, slide projectors and perhaps computer-generatedimages. For example, photographs of video monitor images showing aweapon or other threat have been used to educate system operators on howa threatening object might appear on the system video monitor. Afterreceiving such classroom training, system operators have then typicallybeen tested in the same classroom setting. However, a classroomenvironment for training and testing is typically very different fromthe actual setting in which operators must detect prohibited items forat least several reasons.

[0006] First, in the classroom environment, the system operatortypically sits in a chair and looks at various photographs and othermaterials. This does not accurately replicate real life conditionswhereby the system operator must continuously watch the system's videomonitor, stop and start the system and deal with disturbances.Furthermore, real life conditions involving a steady stream of peoplepassing through a security checkpoint at an airport, many of whom may behostile because they are late for their flights, are not typicallyreplicated in a classroom environment. So despite a system operator'spassing a classroom test with high marks, that is no guarantee of goodperformance by that operator in the field.

[0007] Second, the classroom environment does not effectively teach ortest the operator's vigilance, i.e., focus and attention span, overtime. As noted above, the system operator's vigilance is integral to theongoing success of the security inspection process. Indeed, highvigilance during the early portion of an operator's shift during whichtime no threatening objects pass through undetected does not excuse adecline in attention and focus during the later portion of thatoperator's shift during which time a threatening object passes through.

[0008] Third, other realities of classroom training further reduce itseffectiveness. To this end, classroom training and testing may exposesystem operators to a smaller variety of images and threats than theoperator would see in real life situations. Furthermore, the expense andlogistics of classroom training generally result in less training andtesting than should occur.

[0009] For these and other reasons, classroom-based training and testingis not an optimum or even reliable method of training, testing and/orcertifying the operators of x-ray inspection systems. In view of theseshortcomings, other training tools have been developed.

[0010] One such tool is Threat Imaging Projection (TIP) technology. WithTIP technology, x-ray images of threat objects are generally merged intothe image of non-threat baggage that are displayed on the system'smonitor. The threat images are selected at random from a library ofvarious types of prohibited items. An operator's performance indetecting threats, missing threats and creating false alarms (detectingwhat is believed to be a threat when there actually is no threat) arethen recorded.

[0011] Initial TIP technology implementations were limited to simplyblending x-ray images of threats into the stream of baggage images. Ifthe operator detected the threat and activated the appropriate control,the threat object was erased and feedback was provided to the operatorindicating that he or she had successfully detected the threat. However,such TIP implementations never adequately worked in the field.

[0012] For example, TIP technology could not be efficiently used atactual checkpoints because the images generated by the TIP technologyfor training purposes would generally be indistinguishable from truethreats. Where the TIP technology was set up to be automatic, i.e.,without input to help distinguish between test and true threats,operators often reacted to test threats as though a true threat actuallyexisted. For example, an operator's detecting a test threat imagerepresenting a bomb has resulted in that operator calling a bomb squad.These types of events resulted in large costs, delays and inconvenience.Where human supervisors were used to overcome problems associated withautomatic testing, the testing method still proved to be cumbersome andineffective due to cost, logistical problems and potential coaching ofthe operator by the supervisor.

[0013] More recently, a TIP process was developed that was somewhat moresuccessful to a limited degree. This more recent process: 1) employedautomatic testing, 2) avoided the above-mentioned costs and delays byproviding sufficient feedback to the operator to avoid the situationwhere a test threat was treated as a true threat and 3) recordedindividual operator performance in detecting threats so as to trackperformance over extended periods.

[0014] However, there still existed no method for using the informationprovided by testing with the use of TIP technology in order to certifyoperators. Indeed, varying uses and interpretation of the data indifferent segments of the security inspection industry results invarying levels of system operator proficiency and hardly any type ofuniform certification method. Accordingly, there exists a serious needfor a structured, uniform process for certifying system operators.

SUMMARY OF THE INVENTION

[0015] The current invention generally involves a method and system forusing TIP technology and the operator testing data it generates in orderto certify operators and thereby assure an acceptable and uniform levelof operator performance. More particularly, a method and system isdescribed in which the operator views x-ray images of objects on ascreening system equipped with TIP technology. The system is capable ofrecording, storing and transmitting individual operator performance datarelating to detections, missed detections or false alarms.

[0016] In another aspect of the invention, the method and system providea means to track the types of threats a system operator has seen anddetected, as well as the difficulty of the threats the operator hasseen. To this end, the system contains a library of various threatimages, e.g., guns, bombs, knives, etc., which are classified accordingto type of threat and difficulty of detection.

[0017] In another aspect of the invention, the method and system providea means for reporting the operator's performance in successfullydetecting test images to the appropriate authority, e.g., civil aviationauthorities, airlines and other pertinent organizations. To this end,the system records the results of the operator's performance in adatabase. These results are then compared to the certification criteriathat may be contained in another database to determine whether or notthe operator should be certified. Results are reported to appropriateauthorities or other organizations.

[0018] In another aspect of the invention, software is provided toaccomplish the foregoing. More particularly, software is provided toencode images and aid in their comparison to real threats. Software isalso provided such that the information contained in theabove-referenced (and other) databases may be accessed and used fortraining and certifying operators.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]FIG. 1 is a perspective view of the system of the currentinvention.

[0020]FIG. 2 is an overview schematic of the system of the currentinvention.

[0021]FIG. 3 is an on-screen view of a system operations menu;

[0022]FIG. 4 is a flowchart/schematic of a preferred embodiment of thetraining or certification process wherein the flow of information to andfrom each element of the invention is illustrated.

[0023]FIG. 5 is a flowchart/schematic of an alternative preferredembodiment similar to the one in FIG. 4, except that certaindatabase(s), software and/or other system elements are located at alocation remote from the inspection system.

[0024]FIG. 6 is a flowchart showing a methodology of the softwareassociated with the method and system of the current invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The preferred embodiments will now be described with respect tothe drawings. To facilitate the description, any numeral identifying anelement in one figure generally represents the same element when used inany other figure. The configurations shown in the figures are forillustrative purposes only, and are not intended to limit the scope ofthe current invention.

[0026] A. Description of System Elements

[0027]FIG. 1 is a perspective view of an x-ray screening system 10 thatincludes a housing 12 containing a screening section 14 through whichobjects to be screened are passed. Screening section 14 may generallycomprise a tunnel through which objects pass, and may include an activearea where objects are X-rayed and tunnel-like extensions on either sideof the active area. The tunnel may reflect different cross-sectionalshapes.

[0028] The housing 12 preferably includes a flat base section 16 so thatthe system 10 may readily rest on a flat surface. A video monitor 18 fordisplaying x-ray images of objects located inside the screening section14, and for displaying electronically inserted threat item images, maybe attached to a top section 20 of the housing 12. The system 10preferably includes a conveyor 22 for transporting objects through thescreening section 14.

[0029] An information input device or means, shown in this embodiment asa keyboard 24, is preferably attached to a front section 26 of thehousing 12. The input means 24 preferably allows an operator to log ontoand off of the system 10, to control the conveyor 22, to indicate whenthe operator believes that a threat item image appears on the monitor18, and/or to perform various other functions. The input means 24 mayalternatively be a touch-screen system, wherein an operator touchesimages on a video screen to perform various operations, avoice-activated system, a mouse whereby icons are clicked onto, or anyother suitable input mechanism to facilitate performing the variousfunctions of the current invention. Alternatively, multiple inputmechanisms may be used in conjunction with one another to allow anoperator to perform various procedures.

[0030] The elements of system 10 may be arranged in several differentconfigurations, and a greater or lesser number of elements may be usedin the system 10. Accordingly, FIG. 1 is not intended to limit theinvention to the specific configuration shown.

[0031]FIG. 2 shows a schematic view of an x-ray screening system 102containing a video monitor 104 that displays x-ray images of objects 108to an operator 106. Certain elements in FIG. 2 are generally similar toelements in FIG. 1 but may be referenced by different numerals.Typically, objects 108 pass through system 102 on a conveyor 110. Asdiscussed below, the current invention involves various databases forproviding, storing and transmitting information. Existing types ofdatabases and means for providing communication between such databasesmay be used with the invention.

[0032] The configuration shown in FIG. 2 is for illustrative purposesonly. For example, FIG. 2 depicts different databases, software andother items separately. However, a number of these items may becombined, e.g., one database may contain different types of informationand/or software. Accordingly, FIG. 2 is not intended to limit theinvention to the specific configuration shown.

[0033] Screening system 102 is preferably equipped with TIP technology112 so that images of various prohibited items may be electronicallyinserted into the normal ongoing stream of non-threat images, e.g.,passenger baggage, of objects 108 viewed on monitor 104. The non-threatimages may be images of actual objects 108, or alternatively, may beelectronically generated images. Thus, an operator may be trained on thescreening system 102 without having to use any real objects. In FIG. 2,TIP technology 112 is simply shown as a box within screening system 102,but it will be apparent to one skilled in the art that suitable softwareand electronics are associated with TIP technology 112.

[0034] Screening system 102 preferably also includes a database 114 tostore general information associated with TIP technology 112. Database114 may also encompass other databases discussed herein, and to thisend, FIG. 2 is for illustrative purposes only. Screening system 102 alsopreferably includes a data transmission means 116 to transmit operatorperformance data (and other data) for data collection and reportingpurposes. In FIG. 2, TIP database 114 and transmission means 116 areshown separately from TIP technology 112, but these items may generallycoincide.

[0035] System 102 also preferably includes TIP library 118 that maycontain images of various prohibited items such as guns, bombs, knives,etc. TIP library 118 preferably contains a database defining the threattypes (i.e., defined as a gun, bomb, etc.) and the difficulty associatedwith detecting each type of threat or each image orientation. Forexample, certain types of weapons such as Improvised Explosive Devices(IDEs) are generally considered more difficult to detect than weaponssuch as handguns and will thus have a higher difficulty rating. TIPlibrary 118 may also contain images of what may be thought of asordinary items, but in reality may be used as weapons, e.g., boxcutters, personal knives, blades for shaving, etc.

[0036] The difficulty associated with detecting a given threat may alsodepend on is its orientation or angle of view as projected on themonitor 104. For example, a handgun is generally considered moredifficult to detect when viewed from behind, wherein it may appear to bea simple rectangular piece of metal, as opposed to when viewed from theside, wherein the well known shape of a handgun is more readilyapparent.

[0037] It is preferred that TIP library 118 contains a sufficient numberof different certification images to prevent memorization by theoperator, which would skew any testing or certification effort. It isalso preferred that the certification images contained in TIP library118 be kept secret or otherwise confidential in order to preventoperator cheating. To this end, the certification images stored in TIPlibrary 118 may be changed from time to time to maintain the integrityof the certification process.

[0038] The images contained in TIP library 118 for certificationpurposes are preferably approved and regulated by the appropriatecertifying body such as the FAA. In this manner, the current inventionmay provide a uniform certification process for system operators. Theimages are also preferably graded according to difficulty of detection.For example, the FAA currently recognizes various threat classes andeach class is generally associated with a certain difficulty level. Asdiscussed in more detail below, the invention contemplates that anoperator would need to be able to sufficiently detect images from eachlevel of difficulty in order to become certified.

[0039] The database in TIP library 118 also preferably defines the typeof images that the TIP capability 112 will provide for viewing by theoperator. To this end, images may generally be classified ascertification images or training images. Certification images aregenerally those images used during a certification testing procedure, agiven percentage of which an operator must preferably detect to becomecertified or retain certification. Training images may be similar tocertification images in appearance, and are generally used to trainoperators to detect various threats. However, training images may differfrom certification images in that they portray images of differentthreat objects or images of the same threat object but viewed at adifferent angle or orientation.

[0040] Certification images and training images may be kept separatelyin different libraries. To this end, TIP library 118 may containmultiple sub-libraries in which the respective types of images areseparately contained. Alternatively, two separate TIP libraries (notshown in FIG. 2) may be maintained for this purpose. Maintaining thetraining and certification images separately preferably allows atraining period or certification period to be more readily set up.

[0041] The parameters of a given training or certification period(referred to generally as a testing period) are preferably set by anauthorized user, such as a manager of a scanning company or anappropriate certification authority. These parameters are generallyreferred to as test scripts, because they “script” a test that will beadministered to one or more scanner operators. In a preferred scenario,standardized test scripts may be used across the security industry, orsegments thereof.

[0042] In order to set the test scripts for a given testing period, anauthorized user logs onto the system and enters an ID and passwordprovided by the appropriate authorities. After the authorized user logson, a menu preferably appears, such as the on-screen menu shown in FIG.3, from which the user may select a desired operation, which, in thecase of setting test scripts, may be a ‘TIP Configuration’ operation190. Other operations may include system maintenance 191,importing/exporting databases 192, uploading new TIP images 193,downloading TIP reports 194, viewing TIP reports 195, viewing accessreports 196, and any other suitable operations described herein. A ‘logout’ option 197 is also preferably included on the menu. Once the ‘TIPConfiguration’ operation is chosen, the authorized user may take thefollowing steps to set the test scripts for a given testing period.

[0043] First, a period unit of ‘DAY’ or ‘MONTH’ may be chosen,preferably from a drop-down list appearing on the video monitor 18. If aperiod unit of DAY is chosen, the start date of the testing period maybe set for the current date or a future date. If a period unit of MONTHis chosen, the start date is preferably set for the first day of thecurrent month or of a future month. If a start date is set that falls ona date prior to these specified acceptable start dates, a warningmessage may be displayed and the user may then modify the start date.

[0044] Next, a period length may be chosen, preferably from a drop-downlist, or by entering a number corresponding to the desired period lengthvia the keyboard 24. The period length is generally the length of timethat the testing period will run. If a period unit of MONTH is chosen,and a period length of ‘two’ is entered, the test will run for twomonths. If a period unit of DAY is chosen, and a period length of ‘two’is entered, the test will run for two days.

[0045] In a preferred embodiment, an option to repeat the test may begiven. If the authorized user, or scanner operator (if the operator isgiven such authority), chooses to repeat the test, then the test may beadministered over the previous period length, or until a new test isscheduled, which preferably overrides the repeat test. For example, ifthe testing period is two months, and the user chooses to repeat thetest, then the test will be repeated over the next two months, or untilthe date that a new test is scheduled to start within that two monthperiod, at which time the new test preferably overrides the repeatedtest. In a preferred embodiment, two testing periods may not overlap oneanother in a given system 10.

[0046] A pass percentage, which indicates the test score that anoperator must achieve to obtain certification as a scanner operator, isalso preferably set, either via a drop-down list or by entering apercentage number via the keyboard 24. The pass percentage may rangefrom 1 to 100. Additionally, a description including any informationrelevant to a particular test may optionally be entered by theauthorized user via the keyboard 24.

[0047] The library containing certification images may also containinformation pertaining to the detail of, or difficulty level inidentifying, each certification image. The library containing trainingimages may generally contain more images to allow operator training witha variety of images on which to gain experience. The library containingtraining images may not necessarily specify difficulty levels associatedwith each training image.

[0048] A certification image list for a given test, or set of tests, maybe created or “scripted” by selecting threat item images from a generalsource threat list and moving or dragging the selected items to thecertification image list via a mouse connected to the keyboard 24, orvia any other suitable method. In a similar manner, threat items may beremoved from the certification image list and placed back into thegeneral source threat list. Once the test scripts for a given test, orset of tests, are established, the user may save the test scripts in thesystem memory by clicking on a ‘Save Test’ icon displayed on the videomonitor 18, or by any other suitable data saving method.

[0049] Training images may be used during a certification test, i.e.,they may be randomly merged into the stream of baggage images, to keepoperators alert and prevent them from memorizing which images arecertification images. This generally allows system 102 to provide a mixof images having varying difficulty levels. By providing both trainingand certification images to the operator during certification, theoperator preferably does not rely solely on his or her memory of whatthe certification images may look like. However, training images arepreferably not used as images that will actually count towardscertification since operators may come to memorize training imagesduring the training process.

[0050] Screening system 102 also preferably includes an operatorperformance database (OPD) 120. OPD 120 preferably containscomprehensive records of test results and performance data for eachindividual operator 120 that operates system 102. Information that maybe stored in OPD 120 may include the date and time of each test, thetype of image (along with its difficulty) presented to the operator, andthe operator's test results, including whether the operator achieved ascore at or above the specified pass percentage. To this end,information regarding the operator's 106 detections, non-detections, andfalse alarms, as well as the amount of time an operator spent evaluatinga particular image(s), may be stored in OPD 120. Grades for theoperator's 106 performances may also be stored in OPD 120. It should benoted that other information pertaining to operator performance may alsobe stored in OPD 120.

[0051] There may be different levels or classes of certificationrequiring various different levels of proficiency and/or experience.Accordingly, the operator's 106 levels of proficiency and experienceover time may also be stored in OPD 120. OPD 120 thus preferablyprovides information on the operator's progress over time. This allowsevaluation of how an operator or group of operators perform or progressover time.

[0052] Where applicable, system 102 may also serve to promote anoperator 106 to a higher level of certification by virtue of theinformation in OPD 120 reflecting that operator's performance. In thismanner, system operator 106 may be allowed to operate system 102 attimes or during conditions requiring this higher level of certification.To this end, system 102 may compare the operator's log in information tothe information stored in OPD 120 to determine whether the operator isindeed certified to operate system 102 at that time or during theexisting conditions. Accordingly, the current invention preferablyprovides a safety check to ensure that properly certified operators areoperating the system 102 at appropriate times.

[0053] Screening system 102 also preferably includes a certificationcriteria database (CCD) 122 which may generally contain and set forthrequirements for operator certification. To this end, CCD 122 may setforth different sets of requirements that are required to obtaindifferent types of certification. As such, CCD 122 may specify thenumber of images, types of images and levels of difficulty of imagesthat must be successfully detected by an operator 106 during acertification test. CCD 122 may specify the length of time over which acertification test is to be conducted or the maximum time allowed for anoperator 106 to complete a certification test. CCD 122 may set forth thefrequency of testing necessary to maintain certification. CCD 122 mayalso specify the criteria for determining whether an operator'sperformance constitutes an acceptable/certifiable performance (successcriteria).

[0054] It is preferred that CCD 122 set forth requirements that areuniform for a given industry or application. For example, withcertification for airport baggage scanner operators, it is preferredthat criteria contained in CCD 122 be uniformly used to ensure a minimumcertification level throughout the industry. Such uniform criteria maybe specified by the appropriate authority such as the FAA or other civilaviation authority.

[0055] The criteria contained in CCD 122 may also be changed over timeto reflect changes in desired certification requirements. For example,should a new type of weapon come into being, the certification criteriastored in CCD 122 may be revised to include that image as part of theimages to be seen by operators during the certification process. In thissituation, TIP library 118 may also be updated to include this new typeof image as a training and/or certification image.

[0056] System 102 also preferably includes software 130 that accessesinformation contained in the various databases described above and thatanalyzes operator performance during testing and certification. In FIGS.1-2, software 130 is referred to as Threat Image Projection -Certification Version software (TIP CV), though this name is notintended to limit the scope of the invention. More particularly,certification software 130 may access information in the variousdatabases, receive and record information about operator performance,analyze operator performance and provide reports thereon.

[0057] To accomplish the foregoing objectives, certification software130 may perform various functions. For example, software 130 may controlwhich images are presented to the operator 106 from TIP library 112.This may be accomplished via the test scripts inputted by a user, asdescribed above, or through a random selection of images. It ispreferred that a sufficient number of images from varying levels ofdetection difficulty are presented to the operator so as to render thetraining and/or certification meaningful. To this end, software 130 mayretrieve certification images from TIP library 112 according to thecriteria set forth in CCD 122. And as mentioned above, these criteriamay change over time.

[0058] Certification software 130 may also control the timing of theimages presented to the operator 106, via the test scripts inputted by auser or via a random timing method. In other words, software 130 maycontrol the timing of when various training or certification test imagesare inserted into the normal stream of images viewed by the operator106. For example, software 130 may insert several training orcertification test images in a row, near each other or at a time apartfrom each other. This preferably enhances the training and certificationcapability of system 102 by simulating the randomness by which threatsmay pass through system 102 in real life.

[0059] Software 130 may also ensure that the appropriate number ofcertification images are shown to operator 106 within the time periodspecified in CCD 122 for that type of certification. This may occur byincreasing the frequency of test images displayed to the operator as theend of the test period nears, if necessary.

[0060] Software 130 may also record whether an operator 106 successfullydetects or does not detect the training or test images as well as howmuch time an operator 106 spends evaluating these images. Based on suchinformation, software 130 may determine whether a given operator'sperformance is acceptable. For example, for certification purposes,software 130 may determine whether the operator's performance wassufficient for that operator 106 to become certified or maintain his orher certification level.

[0061] Software 130 may also provide a reporting function on theoperator's performance to the operator and/or the appropriateauthorities, as further described below. For training purposes, software130 may provide information on which training images were successfullydetected or not detected, and may also provide recommendations forfurther training so as to increase the operator's performance in thefuture. This type of reporting function may help provide for uniformtraining standards in a given industry.

[0062] For testing and certification purposes, software 130 may providea report indicating the operator's test score and whether the scoresuffices for certification. This may occur by a comparison of theoperator's performance information that may be contained in OPD 120 tothe certification criteria contained in CCD 122. Software 130 may alsoprovide a description of what types of images were successfully detectedor not, and thus provide information on the areas needing further workby the operator. This reporting function may also help serve toestablish uniform testing guidelines in an industry.

[0063] In determining whether an operator should be certified, software130 may supplement the image and associated difficulty level. Instead ofjust considering the image and level of difficulty in and of themselves,software 130 may analyze the entire image displayed to the operatorcontaining the threat image as well as its surroundings. This may occurbecause a certification image that is normally easy to detect may bemade difficult to detect when placed in a certain location within theongoing stream of non-threat baggage. This may also occur when an imageis not projected well on video monitor 104 due to insufficient x-raypenetration of the baggage, the image into which the certification imagewas merged.

[0064] As a result, an operator may fail to detect an otherwise easilydetectable image and may be graded more harshly than he or she shouldhave been. By supplementing the level of difficulty assigned to a givencertification image with an analysis of the context in which that imageis place, the software 130 may more accurately evaluate an operator'sabilities.

[0065] B. Description of Training and Certification Processes

[0066] The process of training and certifying system operators is nowdiscussed in more detail with continued reference to FIG. 2, but alsowith reference to FIGS. 3 and 4. FIGS. 3 and 4 are schematics generallydepicting the training and certification method of the invention. Theconfiguration shown by these figures is not intended to limit the scopeof the invention.

[0067] As indicated above, the system and method of the currentinvention may provide for uniform training and certification throughoutan industry. The current invention also provides for regulation of thecertification process to avoid cheating. And because the informationcontained in the various databases may be changed to reflect changingconditions and requirements facing operators, the current inventionprovides for flexibility in the certification process.

[0068] In one embodiment of the invention, an operator may be randomlytested for certification purposes while generally performing his or herscreening duties. That is, any time the operator is screening baggage orother objects, a test or certification process may occur. This may occurwith or without the operator's knowledge. Where the testing orcertification occurs without the operator's prior knowledge, thesituation is avoided whereby an operator maintains focus and attentiononly when a known testing period is occurring.

[0069] However, there may be situations where the operator is advised inadvance that the screening system 102 will be providing images to theoperator and/or recording operator performance. For example, system 102may provide training images so that the operator may be trained on howto detect or not detect certain threats. In this situation, it may bepreferred to advise the operator in advance that the system will beproviding training images in order to allow effective and constructivetraining. Once sufficient training has occurred, however, the operatormay be randomly tested without prior knowledge that a test is to occur.

[0070] An operator may log into screening system 102 as shown in step160 a at the beginning of his or her screening shift, training period,testing period or other period during which the operator is operatingsystem 102. Suitable identification and password information may beissued to each operator for the logging in process. In this manner,system 102 may record the identity of the operator who is performing thescreening duties at a particular time. The identification and passwordmethod of logging in also helps to avoid cheating in the training andcertification process. For example, it preferably prevents a moreskilled operator from filling in for a less skilled operator. In similarfashion, operator 106 may log off from system 102 as shown in step 160b.

[0071] Upon an operator logging on to system 102, software 130 mayacknowledge the identity of the operator who will be viewing the images.In this manner, information in OPD 120 may determine whether theoperator logging at the time is certified to operate system 102. Itshould be noted that it is not absolutely necessary for software 130 toacknowledge the operator's identity for training or testing to occur.Also, it is not absolutely necessary for the operator to log on forsoftware 130 to provide images. Accordingly, it is not intended that theinvention be limited as such.

[0072] Some or all of the screening systems 102 in a given facility orfacilities, may be networked so that a given operator 106 can log on toany screening system 102 on that network. Some or all of the screeningsystems 102 may have access to the relevant databases so that a trainingsession, certification test or portion of a certification test could beconducted at different screen system 102 locations. This its providesflexibility in that a given operator could be assigned to differentscreening systems 102 during a certification test without affecting thecertification process. Similarly, the problem of having only onescreening system 102 in a facility that is capable of certifying a givenoperator 106 is avoided, as is the situation where operators maintainfocus and attention only when stationed at that particular system 102.

[0073] Software 130 may provide training and/or testing images dependingon the circumstances. For example, software 130 may access CCD 122 todetermine which images should be shown according to the specifiedcriteria, and then access TIP library 118 to access the desired imagesto provide to operator 106. As shown in FIG. 4, the portion of software130 that interfaces with TIP library 118 is generally referred to asimage control software 130 a.

[0074] When system 102 is being used for training purposes, imagecontrol software 130 a may access a broad range of images which allowthe operator to become familiar with all types of images that might beencountered later on during testing and certification, as well as inreal life. The length of a training session may be controlled bysoftware 130, preferably via the ‘period unit’ and ‘period length’ testscripts that are entered by an authorized user, as described above. Inthis manner, system 102 may be able to correlate the amount of trainingwith the operator's later success when being tested.

[0075] In the testing environment, it may be preferred thatcertification testing of an operator 106 occurs over a set interval oftime whereby test images are provided for viewing on monitor 106. Thelength of the test interval may be specified in the certificationcriteria database 122, or via the ‘period unit’ and ‘period length’ testscripts entered by an authorized user. In this manner, testing forcertification may achieve better uniformity because operators that arenot efficiently able to correctly identify test images will not havesome indefinite period of time to do so which itself would indicate thatthe operator is not worthy of certification.

[0076] When system 102 is being used for certification purposes, theimages accessed by image control software 130 a and provided to theoperator 106 may be determined by the criteria contained in CCD 122. Theimages to be provided to operator 106 may be set forth in a pre-selectedlist or script file which may be randomized by software 130 a. Thispreferably helps to prevent memorization and/or sharing of the specifictest sequence of images.

[0077] Software 130 may also generate automatic test sequences usingcriteria specified in CCD 122 such as the types of images, number ofimages, frequency of projection and length of test. That is, criteriamay be used that call for certain types of images, a certain number ofimages and the frequency at which they are shown. Software 130 may thenuse these criteria to automatically generate a test sequence having astring of certification images according to these criteria.

[0078] Automatic generation of test sequences preferably reduces theburden of having to pre-program a certification test sequence of images.Indeed, by creating multiple groups of test sequences with varyingdifficulty levels, system 102 simplifies the task of generating imagelists for appropriate certification levels.

[0079] Once an operator 106 has completed a test sequence, the software130 a may automatically select a different test sequence for the nexttest. These subsequent sequences may be controlled by an appropriatecertification control office or by allowing software 130 a toautomatically generate a random subsequent test sequence that has notbeen previously seen by the particular operator 106.

[0080] As mentioned above, it is preferred that certain images ofcertain difficulties are provided to the operator 106 when testing forcertification purposes. For example, it is preferred that during atesting/certification interval, at least one image of each type of imagerequiring certification appears on the monitor. To this end, imagecontrol software 130 a may first access CCD 122 to retrieve thepertinent certification regulations and protocols, and for instructionson which images to access from TIP library 118.

[0081] Thereafter, image control software 130 a may provide the testingimages to the operator based on these instructions. In addition to thetype of images provided, image control software 130 a may also receiveinstructions on the frequency that each test image will be provided tothe operator 106. In this manner, the test images viewed by the operator106 may be varied to simulate real life conditions.

[0082] As the operator 106 views the monitor 104, images from TIPlibrary 118 appear. In the training situation, it may be that varioustraining images appear one after the other so as to allow the operator106 to become familiar therewith. In the testing and certificationsituation, test images may be inserted into the ongoing stream ofnon-threat baggage or other objects.

[0083] As the operator 106 views the stream of images on monitor 104, heor she will generally detect (170 a), not detect (170 b) or falselydetect (170 c) what are (or are not) believed to be threat images. Theoperator's 106 responses, i.e., detect 170 a, no detect 170 b or falsedetect 170 c, are preferably recorded in the operator performancedatabase 120 via software 130. Software 130 may use this performanceinformation in various ways.

[0084] For example, the information recorded in OPD 120 may be evaluatedby a portion of software 130 that is designed to do so. In FIG. 4, thisportion of software 130 is referred to as certification software 130 b.Overall, certification software 130 b may be used to compare theinformation recorded in OPD 120 regarding the operator's responses toinformation contained in the CCD 122 that may set forth the correctresponses.

[0085] CCD 122 may also set forth the minimum allowable standards for anoperator to become certified or maintain his or her certification. Suchminimums may be set by the appropriate authority. Images that aresuccessfully detected may go towards meeting the minimum allowablestandards. Upon making this comparison, certification software 130 b maygenerally determine whether the operator passes 180 a and thus becomescertified or maintains certification, or whether the operator fails 180b and thus does not become certified or loses certification. To thisend, software 130 b may take into account the circumstances surroundingthe certification images. For example, if unusual difficulty surroundedthe image such that it was more difficult to detect than would otherwisebe, this may be taken into account so that the operator's performance ismore accurately considered.

[0086] As another example, the information recorded by OPD 120 may beused to determine what further training the operator 106 should undergo.For example, software 130 preferably notes all the incorrect responsesrecorded in OPD 120. If it is seen that operator 106 has troubledetecting a certain type of threat image, after the training or testinginterval has occurred, this information may be used to convey to anoperator (or the appropriate authority) that further training isrecommended on that type of image.

[0087] As another example, the information recorded by OPD 120 may beused during a certification interval that is then occurring. To thisend, the certification criteria contained in CCD 122 may specify that acertain number of different types of images must be detected, during thecertification interval for that operator to pass. If, during thecertification interval, the operator 106 fails to detect a certain typeof image, this information may be recorded in OPD 120 and then used bysoftware 130 to instruct the image control software 130 a to provideanother image of that type to the operator 106.

[0088] After a certification interval ends, certification software 130 bgenerally advises whether the operator 106 passes 180 a or fails 180 b.If the operator 106 passes, he or she is preferably notified and thusneed not undergo further testing until some time in the future. Indeed,the operator 106 may not be aware he or she was being tested until beingmade aware that he or she passed. As mentioned above, subsequent testsare preferably scheduled in a quasi-random manner so as to maintainoperator vigilance.

[0089] The scheduling of subsequent tests may occur per the criteria inCCD 122. To this end, certain operators may need to be tested morefrequently than others. Accordingly, it is preferred that system 102keep track of the type of operator that is operating the system so thathe or she is tested frequently enough.

[0090] If operator 106 fails, operator 106 is again preferably notifiedof the failing result. This may result in various events. For example,operator 106 may be placed on some type of probationary status.Alternatively, operator 106 may fail to become certified (or may becomedecertified) and thus be precluded from operating system 102 for thetime being. It may be that several failing grades in subsequent attemptsto become certified, or consistently low scores when attempting tomaintain certification, results in the operator being given other jobassignments.

[0091] As another example, the results recorded in OPD 120 may be usedto determine whether a given operator 106 is permitted to log on andoperate the screening system 102. If an operator has not achieved ormaintained certification, the operator may be prevented from logging onto and operating the screening system absent the intervention of asupervisor. Similarly, if, during the course of a certification test, anoperator 106 fails to detect a certain number of certification imagessuch that the operator 102 cannot pass even with a perfect performancefrom that point on, the operator 106 may be automatically logged out andprevented from further operating the screening system 102 absentintervention by a supervisor.

[0092] As another example, the results recorded in OPD 120 may be usedto create an individualized training regimen for a particular operator106. By determining the strengths and weaknesses of each operator 106based on the type and difficulty of each image that was not detected,software 130 may then be used to formulate a training regimen thatemphasizes those types of images with which operator 106 had difficulty.In other words, system 102 may provide a training program that istailored to address a particular operator's weaknesses. This in turn maylead to a passing score on the next certification attempt, as well as abetter trained operator work force.

[0093] As another example, the results for various operators 106 thatare recorded in OPD 120 may be evaluated to determine trends indetection. To this end, if a significant number of operators 106 havedifficulty detecting a certain type of threat image, this informationmay be used to institute industry-wide training on this type of threat.Other types of trends may also be determined by evaluating theinformation contained in OPD 120.

[0094] To achieve this goal, software 130 preferably provides areporting function by accessing and manipulating data contained in TIPlibrary 118, OPD 120 and/or CCD 122 to provide desired information. Tothis end, software 130 preferably includes various types of filters sothat authorized users may extract certain types of information from thevarious databases in a format that provides useful reports. Such reportsmay include threat detection percentages by category, individualscreener performance summaries, performance comparisons of multiplescreeners, and any other suitable information for evaluating system andoperator performance.

[0095] The ‘threat detection by category’ reports preferably include thefollowing fields:

[0096] the name and ID number of each selected screener;

[0097] the dates on which each screener underwent testing;

[0098] the time within a test at which each image appeared;

[0099] the threat category (e.g., gun, bomb, etc., which will be thesame for each item in a ‘threat detection by category’ report);

[0100] the subcategory of each threat, i.e., conventional or otherwise;

[0101] the threat description (e.g., the specific gun model or bombtype);

[0102] the decision outcome, i.e., whether the threat was identified(“hit” or “miss”);

[0103] the recognition time for each identified item, i.e., the amountof time that elapsed before the screener identified the threat;

[0104] the total recognition time for all the threats from a testingperiod; and

[0105] the average recognition time for all the threats from a testingperiod.

[0106] The ‘individual screener performance summary’ reports preferablyinclude the following fields:

[0107] the name and ID number of a selected screener;

[0108] the airport and terminal where the screener underwent testing;

[0109] the date each report was created;

[0110] the number of shifts worked by the screener;

[0111] the number of bags screened by the screener;

[0112] the number of certification images presented;

[0113] the number of certification images properly identified;

[0114] the number of certification images missed;

[0115] the percentage score of properly identified items;

[0116] the total and average daily performance of the screener; and

[0117] whether the screener passed or failed a given test.

[0118] The ‘screener comparison’ reports preferably include thefollowing fields:

[0119] the name and ID number of each selected screener;

[0120] the number of shifts that each screener worked;

[0121] the number of bags screened by each screener;

[0122] the number of certification images presented to each screener;

[0123] the number of certification images properly identified by eachscreener;

[0124] the number of certification images missed by each screener;

[0125] the percentage score of properly identified items for eachscreener;

[0126] whether each screener passed or failed a given test;

[0127] the grand total of properly identified items for all selectedscreeners; and

[0128] the average performance of all selected screeners as a whole.

[0129] The TIP reports may be downloaded by clicking on the ‘DownloadTIP Reports’, or similar option, from the main menu. The ‘threatdetection by category,’ ‘individual screener performance summary,’‘screener comparison’ and any other suitable reports may be downloadedindividually, or all of the reports may be downloaded together. Todownload the reports, a test period and a destination are preferablyselected by an authorized user. For example, an authorized user mayselect all CV reports administered over a two month period from February1 to March 31 in a given year, and choose to have those reportsdownloaded to a floppy drive, a zip drive, a hard drive, or any othersuitable medium. If a test period and/or a destination is not selected,a warning message will be displayed, and the authorized user may thenadd the required information. The downloaded CV reports are preferablysaved to the selected destination as text (.txt) files.

[0130] Certain aspects of the functionality provided by system 102 thathave been mentioned above bear reiteration in this description of thetesting and certification method. First, certification images arepreferably approved and/or graded for difficulty by the appropriateauthority. Second, certification images are preferably kept secret toavoid cheating by operators seeking certification. Third, it ispreferred that certification images are sufficiently numerous to avoidmemorization by the operators 106. To this end, certification images maybe updated over time, but when this occurs, it is again desired that theappropriate authority approve and/or grade the new images.

[0131] Referring now to FIG. 5, an alternative embodiment of the currentinvention is shown wherein like reference numerals are used except thatnumerals in the 200s are used instead of the 100s as in FIG. 4.

[0132] In this embodiment, the certification software 230 b may bephysically located at a remote location away from screening system 202.Other elements described above may also be located remotely to system102. In this embodiment, software 230 b and other components of system202 may be under the control of, and located at, the appropriateauthority such as the FAA. As discussed below, this may allow theappropriate authority to have more control over evaluating operatorresponses 270 a-c and determining whether operators become certified (ormaintain their certification). This alternative embodiment issubstantially similar to the embodiment of FIG. 4 but may differ asfollows.

[0133] As shown by the doffed line, the operator responses 270 a-crecorded in OPD 220 may be transmitted to certification software 230 blocated at the remote location via a global secure internet connection,a hierarchical network structure, a physical diskette, wirelesstransmission, or some other means of data transmission. Thecertification criteria database 222 may be present at both the locationof the screening system 202 as well as the remote location. In thismanner, CCD 222 a located at the screening system 202 may be used totest an operator 206, while CCD 222 b located at the remote location maybe accessed when determining if an operator passed.

[0134] More particularly, image control software 230 a may access CCD222 a as a factor in determining the timing and sequence of images to bedisplayed on monitor 208 during the certification process at thescreening location. At a separate location, certification software 230 bmay compare the standards contained in CCD 222 b with the responsesrecorded for each individual operator 206 as transmitted from OPD 220 inorder to determine whether the individual operator 206 should becertified 280 a or not certified 280 b. This alternative configurationpreferably allows the governing body more direct control of thecertification process. As shown by the dotted line, updating one of CCD222 a or 222 b may result in the updating of the other of CCD 222 b or222 a.

[0135] Referring now to FIG. 6, the functionality of software 130 andhow it is used with system 102 is now further described. As shown, x-raysystem software 312 may reside on a computer associated with system 102.Software 312 may generally be associated with the system's TIPcapability (112 in FIG. 2) and is thus represented by the referencenumeral 312 in FIG. 6. Certification software 330 (similar to software130 and 230 in FIGS. 1-3) may reside on a server that has connectivityto system 102. Indeed, in a preferred embodiment, the current inventionis used in a network environment.

[0136] Operator 106 may log onto system 102 as shown in step 302. Asshown in step 304, certification software 330 preferably generates acertification test script containing various parameters such as thelength of the testing period, the minimum score required to pass and theimages to be used. During the testing period, some number ofcertification images may be shown to the operator 106 who mustsuccessfully detect some minimum number to pass. As discussed above,these parameters may vary according to criteria contained in CCD 322.Accordingly, there may be connectivity with OPD 320 in FIG. 6 when thetest script is being generated as shown in step 304.

[0137] The test script may also vary according to the type ofcertification sought by operator 106. As also discussed above, the testscript is preferably generated by software 330 automatically.Alternatively, however, test scripts may be programmed by theappropriate authority.

[0138] As shown in step 306, the certification test script 304 may betransmitted to system 102. Various types of transmission may be used,but in one embodiment, transmission may occur electronically over alocal area network (LAN). It should be noted that the foregoingdiscussion is not intended to require that steps 302, 304 and 306 occurin a certain required order every time.

[0139] A certification test may then generally occur. The certificationinterval will begin and over time, the operator 106 may be shown bothcertification images and standard, or training, TIP images. As discussedabove, both types of images may be provided to operator 106 to preventmemorization or sharing of the certification images. At some point afterthe start of the interval, the operator 106 will have seen some portionof overall number of certification images set forth from the test script304.

[0140] A list of remaining certification images to be shown to operator106 (remaining images list or RIL) is preferably generated as shown instep 308. RIL 308 may be generated at various points in the testinginterval. In a preferred embodiment, RIL 308 may be generated each timeoperator 106 logs onto system 102.

[0141] As shown in step 310, queries may be generated as to how manycertification images of the original number remain to be seen byoperator 106, and how many days remain in the certification interval.Other queries may be generated. Based on these two (or other) factors,it is determined whether a certification image or a standard TIP imagewill be next shown to operator 106. To this end, for example, if thereare few days remaining in the certification interval and there are asignificant number of certification images that remain to be seen byoperator 106, it may be that a certification image is more likelyprovided.

[0142] If the outcome of step 310 is that a certification image isprovided as in step 312 a, a certification image is preferably pickedfrom RIL 308 at random as shown in step 314. In connection therewith,that certification image is preferably removed from RIL 308. If theoutcome of step 310 is that a standard TIP image is provided as shown instep 312 b, a certification image will not be removed from RIL 308 andstep 314 is generally bypassed. Upon either a certification TIP image orstandard TIP image being chosen, the TIP image is merged into the imageof the image of the baggage as shown in step 316.

[0143] Thereafter, the operator's performance may be recorded as shownin step 318. The operator performance may generally comprise a detect,no detect or false alarm as discussed in connection with steps 170 a-cin FIG. 4. The information reflecting the operator's performance may betransmitted to operator performance database (OPD) 320 as shown in step330. As discussed above, it is generally preferred that the operator'sperformance on only the certification images count towards determiningwhether certification is obtained or maintained.

[0144] As the certification interval progresses, operator 106 will beshown more images in the manner described above. This repetitive processis shown by step 332. Generally, it is preferred that operator 106 hasthe opportunity to see all certification images during the certificationinterval. Alternatively, it is preferred that operator see at least aminimum number of certification images that would enable the operator tobecome certified.

[0145] As the operator's performance data is stored in OPD 320, it iscompared with the information stored in CCD 322 to determine whether theparticular operator passes. This comparison may occur multiple timesuntil the certification interval is complete. To this end, query 334 maybe generated on whether the testing interval is completed. If not, asshown in step 336 a, additional comparison(s) between operatorperformance data in OPD 320 and certification criteria in CCD 322 willoccur.

[0146] When the certification interval is complete as shown in step 336b, the operator's overall performance will be assessed according to theminimum score requirement set forth in the test script 304. If theminimum score is achieved, operator 106 becomes certified or maintainscertification as shown in step 380 a. Alternatively, if operator 106does not meet the minimum score, certification is not obtained or may belost as shown in step 380 b.

[0147] In an alternative embodiment, the screening system 102 isincorporated into a checkpoint wherein various types of otherinformation generated by other screening methods or devices are recordedto supplement the information described above. In locations such asairports, x-ray screening systems are generally used in combination withmetal detectors, where the x-ray screening system is used to detectthreats contained in baggage and the metal detector is used to detectthreats concealed by people under clothing, etc.

[0148] The alarms on metal detectors may be activated by a significantpercentage of people passing through checkpoints, requiring furtherinspection. Other methods and devices may also be used to detect threatsand the information generated by their use may also be used tosupplement the information described above.

[0149] By recording the number of people and bags that pass through acheckpoint as well as the number of detections by the metal detector, amore complete measure of the total activity at that checkpoint may beascertained. Accordingly, the current invention preferably includesdatabases and software to record, store and report on the volume ofpeople passing through a checkpoint as well as the number of detectionsmade by the metal detector(s) (or other detection devices) at thatcheckpoint. It is also preferred that the metal detector and/or otherdetector information may be transmitted over a network in similarfashion to the certification information described above. This otherdetection information may be incorporated with the certificationinformation to provide an overall picture of operator performance.

[0150] This activity may be compared with operator performance data fromthe OPD 120 to determine the effect of overall checkpoint activitylevels on operator performance. For example, operator performance maysuffer during periods of increased activity at a checkpoint due todistraction. As such, this information may be used to determine aminimum number of personnel at a given checkpoint for given activitylevels in order to maintain a specified level of threat detection byoperators 106.

[0151] Information relating to the amount of activity at a checkpointmay also be used to determine the timing of certification testing andthe timing of the display of threat images during certification testing.For example, certification testing may be avoided during periods of highactivity because of delays it may cause. Similarly, the number ofcertification images displayed during times of increased activity may bedecreased in order to lessen delay and increase the flow of peoplethrough a checkpoint. Alternatively, the number of certification imagesdisplayed during periods of increased activity may be increased in orderto test and certify an operator's performance under more difficultconditions.

[0152] While various preferred embodiments and configuration have beendescribed above, the current invention is not limited to these exactembodiments and configuration. For example, various databases orsoftware components separately described above may be combined.Accordingly, one skilled in the art will appreciate that variations maybe made to the foregoing description without departing from the scope ofthe current invention.

What is claimed is:
 1. A system for certifying security inspectionsystem operators, comprising: an x-ray inspection machine for inspectingbaggage items; a threat item image database including a plurality ofcertification images of which an operator must properly detect apredetermined percentage to obtain a passing certification score; amonitor for displaying threat item images from the threat item imagedatabase, and for displaying x-ray images of baggage items; software foraccessing the threat item image database to electronically produce athreat item image such that it appears on the monitor as if a threatitem is located inside a baggage item; and an input device for allowingthe operator to indicate when the operator believes that a threat itemimage appears on the monitor.
 2. A system according to claim 1 whereinthe threat item image database further includes training images, thedetection of which do not factor into calculating the certificationscore.
 3. A system according to claim 2 wherein the software randomlyselects certification images and training images from the threat itemimage database for displaying on the monitor.
 4. A system according toclaim 2 wherein the certification images and the training images arestored in separate libraries within the threat item image database.
 5. Asystem according to claim 1 further comprising an operator performancedatabase accessible by the software for storing operator performancerecords.
 6. A system according to claim 1 further comprising acertification criteria database accessible by the software containingrequirements for operator certification.
 7. A system according to claim1 further comprising a conveyor belt for moving baggage items into andout of the x-ray machine.
 8. A system according to claim 1 wherein thecertification images include images of at least one of bombs, knives,guns, and improvised explosive devices.
 9. A system according to claim Iwherein each certification image and an orientation angle of eachcertification image is associated with a detection difficulty level thatis used in calculating the certification score.
 10. A system accordingto claim I wherein the x-ray images of baggage items are images ofactual baggage items located inside the x-ray machine.
 11. A systemaccording to claim 1 wherein the x-ray images of baggage items areelectronically generated by the software.
 12. A system according toclaim 1 further comprising reporting means in communication with theprocessor for reporting an operator certification score to an operatorcertification authority such that the certification score may beevaluated.
 13. A system for uniform certification of security inspectionsystem operators, comprising: an x-ray inspection machine for inspectingbaggage items; a threat item image database including a plurality ofstandardized certification images of which an operator must properlydetect a predetermined percentage to obtain a passing certificationscore; a monitor for displaying threat item images from the threat itemimage database, and for displaying x-ray images of baggage items locatedinside the x-ray inspection machine; software for accessing the threatitem image database to electronically produce a threat item image suchthat it appears on the monitor as if a threat item is located inside abaggage item; an input device for allowing the operator to indicate whenthe operator believes that a threat item image appears on the monitor;an operator performance database accessible by the software for storingoperator performance records; and a certification criteria databaseaccessible by the software containing operator certificationrequirements.
 14. A system according to claim 13 wherein the softwarecompares an operator performance record to the operator certificationrequirements to determine whether an operator has met requirements forcertification.
 15. A system according to claim 13 further comprising adata transmission means for transmitting operator performance dataobtained during operator testing to the operator performance database.16. A system according to claim 13 wherein at least one of the software,the operator performance database, and the certification criteriadatabase is located at a location remote from the x-ray inspectionmachine.
 17. A system according to claim 13 further comprising datacollection means in communication with the software for collecting dataused to create the operator performance records.
 18. A system accordingto claim 13 wherein the threat item image database further includestraining images, the detection of which do not factor into calculatingthe certification score.
 19. A system according to claim 18 wherein thesoftware is programmable to display a pre-determined set of trainingimages to meet specific training needs of a given operator.
 20. A systemaccording to claim 13 wherein the software is programmable to displayeach certification image at least once during certification testing ofan operator.
 21. A method of certifying operators of security inspectionsystems, wherein baggage items are passed through an x-ray inspectionmachine, comprising the steps of: displaying x-ray images of the baggageitems on a monitor for viewing by an operator; electronically insertingcertification images of threat items into a selection of the x-rayimages of baggage items; providing an input device for allowing theoperator to indicate when the operator believes that a threat item imageappears on the monitor; determining a number of certification imagesthat the operator properly detected; and calculating an operator testscore.
 22. The method of claim 21 further comprising the steps ofdetermining the number of times that the operator incorrectly identifiedan image as a threat item image, and factoring that number into theoperator test score.
 23. The method of claim 21 further comprising thestep of randomly electronically inserting training images of threatitems into a selection of the x-ray images of baggage items, thedetection of which do not factor into calculating the operator testscore.
 24. The method of claim 21 wherein a random selection ofcertification images are inserted into a selection of the x-ray imagesof baggage items.
 25. The method of claim 21 wherein a pre-determinedset of certification images are inserted into a selection of the x-rayimages of baggage items.
 26. The method of claim 21 wherein the operatortest score is calculated over a test period of several days.
 27. Themethod of claim 26 wherein each of the certification images is displayedon the monitor at least once during the test period.
 28. The method ofclaim 26 wherein the test period has a duration of at least 120 days.29. The method of claim 21 further comprising the step of reporting theoperator test score to an operator certification authority forevaluation of the operator test score.
 30. The method of claim 21further comprising the steps of recording detection performance of theoperator in a performance record, and storing the performance record inan operator performance database.
 31. The method of claim 30 wherein theperformance record includes data relating to the number of times thatthe operator correctly identified a certification image, the number oftimes that the operator incorrectly identified an image as a threat itemimage, and the number of times that the operator failed to detect acertification image.
 32. The method of claim 30 wherein the step ofcalculating the operator test score comprises comparing the operatorperformance record to certification requirements stored in acertification criteria database.
 33. A method of testing operators ofsecurity inspection systems, wherein an operator must detect apredetermined percentage of certification images of threat items toobtain certification, comprising the steps of: displaying x-ray imagesof baggage items on a monitor to be viewed by the operator;electronically inserting images of threat items into a selection of thex-ray images of baggage items; providing an input device for allowingthe operator to indicate when the operator believes that a threat itemimage is detected on the monitor; determining how many certificationimages the operator properly detected; calculating an operator testscore; and reporting the operator test score.
 34. The method of claim 33further comprising the step of running actual baggage items through anx-ray machine to generate the x-ray images of baggage items.
 35. Themethod of claim 33 further comprising the step of electronicallygenerating the x-ray images of baggage items.
 36. The method of claim 33further comprising the steps of instructing the operator to enter apersonal password before operating an x-ray inspection system, andpreventing the operator from operating the x-ray inspection system untila proper password has been entered.
 37. The method of claim 33 furthercomprising the step of storing a performance record of the operator inan operator performance database.
 38. The method of claim 37 wherein theperformance record includes data relating to the number of times thatthe operator correctly identified a certification image, the number oftimes that the operator incorrectly identified an image as a threat itemimage, and the number of times that the operator failed to detect acertification image.
 39. The method of claim 37 further comprising thestep of comparing the performance record to operator certificationrequirements stored in a certification criteria database to determinewhether the operator has met the certification requirements.
 40. Themethod of claim 33 wherein the electronically inserted threat itemimages include only certification images.
 41. The method of claim 33wherein the electronically inserted threat item images includecertification images and training images that are randomly inserted intothe x-ray images of baggage items.
 42. The method of claim 41 whereinthe detection of training images by the operator does not factor intocalculating the operator test score.
 43. A method of testing operatorsof security inspection systems over a test period of a pre-determinednumber of days, wherein an operator must detect a predeterminedpercentage of certification images of threat items during the testperiod to obtain certification, comprising the steps of: displayingx-ray images of baggage items on a monitor to be viewed by the operator;electronically inserting a random selection of certification images intoa selection of the x-ray images of baggage items; electronicallyinserting training images into a selection of the x-ray images ofbaggage items; increasing the frequency at which the certificationimages are inserted as the test period nears an end, to ensure that eachof the certification images is inserted at least once during the testperiod; providing an input device allowing the operator to indicate whenthe operator believes that a threat item image is detected on themonitor; determining how many certification images the operator properlydetected; calculating an operator test score based on the number ofproperly detected certification images, the number of certificationimages not detected, and the number of times that the operatorimproperly identified an image as a threat image.
 44. The method ofclaim 43 further comprising the step of reporting the operator testscore to an operator certification authority so that the test score maybe evaluated to determine whether the operator qualifies forcertification.
 45. The method of claim 44 wherein the reporting stepoccurs before the test period ends if the operator can no longer detecta sufficient number of certification images during the remainder of thetest period to achieve at least the predetermined percentage to obtaincertification.
 46. The method of claim 43 further comprising the step ofensuring that each certification image is electronically inserted intothe x-ray images of baggage items a predetermined number of times. 47.The method of claim 46 wherein specified certification images areinserted more times than other certification images.
 48. The method ofclaim 43 wherein each certification image and/or an orientation angle ofeach certification image is associated with a detection difficulty levelthat is used as a factor in calculating the operator test score.
 49. Themethod of claim 48 further comprising the steps of tracking andrecording the types of certification images, and their associateddifficulty levels, that have been inserted into the x-ray images ofbaggage items during the test period.
 50. The method of claim 48 furthercomprising the step of requiring the operator to properly detect atleast one certification image from every difficulty level to obtain apassing test score.